Preventing transition shocks during transitions between realities

ABSTRACT

A method, system and computer program product for transitioning between different realities. The distance of the objects from a reference point in the first and second reality environments is determined. For example, the distance of the object&#39;s location in the first reality environment from a reference point (e.g., camera system) as well as the distance of the object&#39;s location in the second reality environment from the reference point is determined. A determination is made as to whether the likelihood of a transition shock event (e.g., motion sickness) occurring as a user transitions between the first and second reality environments exceeds a threshold value. In response to the likelihood of the transition shock event occurring exceeding the threshold value, the distance of the object from the reference point in the first and/or second reality environments is modified so as to prevent users from experiencing transition shocks when transitioning between realities.

TECHNICAL FIELD

The present invention relates generally to virtual reality and augmentedreality systems, and more particularly to preventing transition shocks(e.g., disorientation, nauseousness, motion sickness) during transitionsbetween realities (e.g., transitioning between virtual reality andaugmented reality environments).

BACKGROUND

Virtual reality (VR) is a computer technology that uses virtual realityheadsets or multi-projected environments, sometimes in combination withphysical environments or props, to generate realistic images, sounds andother sensations that simulate a user's physical presence in a virtualor imaginary environment. A person using virtual reality equipment isable to “look around” the artificial world, and with high quality VRmove around in it and interact with virtual features or items. Theeffect is commonly created by VR headsets consisting of a head-mounteddisplay with a small screen in front of the eyes, but can also becreated through specially designed rooms with multiple large screens.

Augmented reality (AR) is a live direct or indirect view of a physical,real-world environment whose elements are “augmented” bycomputer-generated perceptual information, ideally across multiplesensory modalities, including visual, auditory, haptic, somatosensory,and olfactory. The overlaid sensory information can be constructive(i.e., additive to the natural environment) or destructive (i.e.,masking of the natural environment) and is spatial registered with thephysical world such that it is perceived as an immersive aspect of thereal world environment (also referred to herein as the “real reality”).In this way, augmented reality alters one's current perception of a realworld environment, whereas, virtual reality replaces the real worldenvironment with a simulated one.

Currently, when a user transitions between different realities, such asbetween the virtual reality and the augmented reality environments orbetween the virtual reality and the real reality environments, the usermay experience disorientation, motion sickness and/or nauseousnessthereby degrading the user's experience.

SUMMARY

In one embodiment of the present invention, a method for transitioningbetween different realities comprises determining a distance of anobject from a reference point in a first reality environment and in asecond reality environment. The method further comprises ascertaining,by a processor, that a likelihood of a transition shock event occurringexceeds a threshold value as a user transitions between the first andsecond reality environments, where the transition shock event is anoccurrence of adverse symptoms experienced by the user as the usertransitions between the first and second reality environments. Themethod additionally comprises modifying, by the processor, the distanceof the object from the reference point in one or more of the firstreality environment and the second reality environment in response toascertaining that the likelihood of the transition shock event occurringexceeds the threshold value.

Other forms of the embodiment of the method described above are in asystem and in a computer program product.

The foregoing has outlined rather generally the features and technicaladvantages of one or more embodiments of the present invention in orderthat the detailed description of the present invention that follows maybe better understood. Additional features and advantages of the presentinvention will be described hereinafter which may form the subject ofthe claims of the present invention.

BRIEF DESCRIPTION OF THE DRAWINGS

A better understanding of the present invention can be obtained when thefollowing detailed description is considered in conjunction with thefollowing drawings, in which:

FIG. 1 illustrates a virtual reality/augmented reality/real realitysystem configured in accordance with an embodiment of the presentinvention;

FIG. 2 illustrates an embodiment of the present invention of a hardwareconfiguration of the system controller which is representative of ahardware environment for practicing the present invention;

FIG. 3 is flowchart of a method for storing information pertaining touser actions and experiences in transitioning between realities whichwill be used to determine the likelihood of a transition shock eventoccurring in accordance with an embodiment of the present invention;

FIG. 4 is a flowchart of a method for preventing transition shocksduring transitions between realities in accordance with an embodiment ofthe present invention; and

FIG. 5 illustrates a record detailing the modification of the distanceof the object(s) from the reference point in the first realityenvironment and/or the second reality environment so that the user willbe less likely to experience a transition shock in accordance with anembodiment of the present invention.

DETAILED DESCRIPTION

The present invention comprises a method, system and computer programproduct for transitioning between different realities. In one embodimentof the present invention, the distance of objects from a reference pointin the first and second reality environments is determined. For example,the distance of the object's location in the first reality environmentfrom a reference point (e.g., camera system) as well as the distance ofthe object's location in the second reality environment from thereference point is determined. A determination is made as to whether thelikelihood of a transition shock event occurring as a user transitionsbetween the first and second reality environments exceeds a thresholdvalue. A “transition shock event,” as used herein, refers to theoccurrence of adverse symptoms, such as disorientation, nauseousness ormotion sickness, experienced by the user when the user transitionsbetween different realties. In response to the likelihood of thetransition shock event occurring exceeding the threshold value, thedistance of the object from the reference point in the first and/orsecond reality environments is modified so as to prevent users fromexperiencing transition shocks (e.g., disorientation, nauseousness,motion sickness) when transitioning between realities (e.g.,transitioning between the virtual reality and augmented realityenvironments).

In the following description, numerous specific details are set forth toprovide a thorough understanding of the present invention. However, itwill be apparent to those skilled in the art that the present inventionmay be practiced without such specific details. In other instances,well-known circuits have been shown in block diagram form in order notto obscure the present invention in unnecessary detail. For the mostpart, details considering timing considerations and the like have beenomitted inasmuch as such details are not necessary to obtain a completeunderstanding of the present invention and are within the skills ofpersons of ordinary skill in the relevant art.

While the following discusses the present invention in connection withtransitioning between different realities, such as virtual reality,augmented reality and real reality environments, the principles of thepresent invention may be applied to transitioning between otherrealities, such as a mixed reality environment (merging of real andvirtual worlds to produce new environments and visualizations wherephysical and digital objects co-exist and interact in real time). Aperson of ordinary skill in the art would be capable of applying theprinciples of the present invention to such implementations. Further,embodiments applying the principles of the present invention to suchimplementations would fall within the scope of the present invention.

Referring now to the Figures in detail, FIG. 1 illustrates a virtualreality/augmented reality/real reality system 100 configured inaccordance with an embodiment of the present invention. System 100 isconfigured to allow a user 101 to transition between differentrealities, such as transitioning between the virtual reality, theaugmented reality and real reality environments. In one embodiment, user101 may wear a virtual reality/augmented reality (VR/AR) headset 102that includes a display 103 providing a graphical environment for VRgeneration and/or AR generation. The graphical environment includesgraphical images and/or computer-generated perceptual information.Display 103 encompasses part or all of a user's field of view.

Exemplary embodiments of headset 102 include a visor, a helmet, goggles,glasses and other similar arrangements. Examples of VR/AR headsets 102include Oculus Rift® from Oculus VR®, VIVE® from HTC® Corporation andGear VR® by Samsung® Electronics. A further example of VR/AR headset 102presently available is the HoloLens® from Microsoft®. Furthermore, inone embodiment, headset 102 may include any one or more of thefollowing: headphones to provide auditory feedback, vibration means toprovide vibration feedback, and other sensors placed on or around theforward facing surface when in use.

System 100 further includes a system controller 104 in communicationwith display 103 of headset 102 via a network 105, which is capable ofgenerating and/or controlling the graphical environment. Systemcontroller 104 may provide instructions to display 103, for example, byexecuting program code including an application program related to theVR and/or AR simulation.

In one embodiment, system controller 104 is a computing device, such asa mobile computing device (e.g., smartphone), which includes a cameraand MEMS (microelectromechanical systems) sensors, such as anaccelerometer, GPS (Global Positioning System) and a solid statecompass, thereby enabling system controller 104 to be a suitable ARplatform. In other embodiments, system controller 104 is a home videogame console system, a set-top box or a media player. A description ofthe hardware configuration of system controller 104 being a mobilecomputing device is provided below in connection with FIG. 2. It isnoted, however, that system controller 104 is not to be limited in scopeto being a mobile computing device as discussed above.

Network 105 may be, for example, a wireless wide area network, a GlobalSystem for Mobile Communications (GSM) network, a Wireless ApplicationProtocol (WAP) network, a WiFi network, an IEEE 802.11 standardsnetwork, various combinations thereof, etc. Other networks, whosedescriptions are omitted here for brevity, may also be used inconjunction with system 100 of FIG. 1 without departing from the scopeof the present invention.

Additionally, system 100 may include a user interface device 106 (may beseparate from system controller 104 as shown in FIG. 1 or embodied asbeing part of system controller 104), through which user 101 interfaceswith system controller 104. In particular, user 101 may interact withthe application program related to the VR and/or AR simulation via userinterface device 106.

Furthermore, system 100 includes a smart wearable 107 (e.g., smartwatch)worn by user 101, such as on the user's wrist. In one embodiment, smartwearable 107 is configured to monitor the user's heart rate, the user'sbody temperature, levels of a user's sweat, etc. Such monitoredinformation may be transmitted to system controller 104, such as vianetwork 105. Such monitored information is one of the means formonitoring the user's experience in transitioning between realities asdiscussed further below. For example, such monitoring may includeinformation that indicates that user 101 experienced a “transitionshock” when transitioning between realities. A “transition shock,” asused herein, refers to the occurrence of adverse symptoms, such asdisorientation, nauseousness or motion sickness, experienced by user 101when user 101 transitions between different realties. Such symptoms maybe determined based on the information acquired by smart wearable 107.For example, a high heart rate may indicate nausea or motion sickness.Cold sweating (sweating when the body temperature is not high) mayindicate motion sickness. Excessive sweating may indicatedisorientation.

Additionally, system 100 includes an object tracking system 108connected to system controller 104 via network 105. Object trackingsystem 108 is configured to track objects in the field of view of user101 and track the actions of user 101, such as tracking the user'shands, arms, legs, feet, head and torso. An “object,” as used herein,refers to something material (whether graphical image or physical) thatcan be perceived by a user's senses. Object tracking system 108 may beused to provide a real time image stream to display 103 of headset 102,for example, an image stream depicting an avatar of user 101 and/or ofother objects in the field of view of user 101 in the virtual realityenvironment.

In one embodiment, object tracking system 108 includes a camera system109, including one or more cameras, such as a two-dimensional camera, athree-dimensional camera, an infrared camera and a time of flight (ToF)camera. Camera system 109 may include complementarymetal-oxide-semiconductor (CMOS) or charged-coupled device (CCD) imagesensors, or any other form of sensor to form images. An example ofcamera system 109 is the Pro C920 or C930 Full HD by Logitech®. In oneembodiment, camera system 109 may include a camera with an infraredfilter, which can be used for object tracking. In one embodiment, camerasystem 109 may include an RGB camera, which can be used for generationof augmented reality simulations. In one embodiment, different frames ofa single camera may be processed in an alternating manner, e.g., with aninfrared filter and for RGB, instead of separate cameras. Images of morethan one camera may be stitched together to give a field of viewequivalent to that of the user.

In one embodiment, camera system 109 may be mounted to or integrated aspart of headset 102, or otherwise adapted to be worn on the head of user101, whereby the capture area of the camera is arranged to record anequivalent to a field of view of user 101 when wearing headset 102.Additional cameras may be arranged elsewhere to track other parts of thebody of user 101. Use of additional camera(s) to cover areas outside theimmediate field of view of user 101 (or outside the field of view ofother cameras within the system) may provide the benefit of allowingpre-rendering (or earlier initiation of other calculations) involvedwith the augmented or virtual reality rendition of those areas.

In one embodiment, system controller 104 implements an applicationprogram for determining from the image stream provided by camera system109 the arrangement of objects in the capture area. For object tracking,the application program may implement known techniques, such as featureextraction and identification. For example, one such technique is thespeed-up robust features (SURF) algorithm. In one embodiment, objecttracking system 108 may include an ultrasonic, magnetic, or other fieldtracker, which may be mounted on headset 102, which is able to sense amagnetic field generated from a local base station, video game consoleor other apparatus. The magnetic field tracker provides informationabout the sensed magnetic field to an application program, which derivesthe position and orientation of the magnetic field tracker, and thusheadset 102, relative to a base station or the magnetic field trackerfrom such information.

System 100 further includes a database 110 connected to systemcontroller 104. In one embodiment, database 110 is configured to storeinformation pertaining to the monitored user actions and experience whenuser 101 transitions between realities as well as stores the changes indistances among the objects as well as the changes in distances betweenthe objects and a reference object when user 101 transitions betweenrealities as discussed further below. Furthermore, database 110 may beconfigured to store the transition speed when user 101 transitionsbetween realities as discussed further below. Furthermore, as discussedfurther below, the monitored user experience in transitioning betweenrealities may be mapped to the changes in the distances among theobjects and/or between the objects and a reference object (as well asthe transition speed) when user 101 transitions between realities, andsuch mapping may be stored in database 110.

System 100 is not to be limited in scope to any one particulararchitecture. System 100 may include any number of users 101, headsets102, displays 103, system controllers 104, networks 105, user interfacedevices 106, smart wearables 107, object tracking systems 108, camerasystems 109 and databases 110.

Referring now to FIG. 2, FIG. 2 illustrates a hardware configuration ofsystem controller 104 (FIG. 1) which is representative of a hardwareenvironment for practicing the present invention. Referring to FIG. 2,system controller 104 has a processor 201 coupled to various othercomponents by system bus 202. An operating system 203 runs on processor201 and provides control and coordinates the functions of the variouscomponents of FIG. 2. An application 204 in accordance with theprinciples of the present invention runs in conjunction with operatingsystem 203 and provides calls to operating system 203 where the callsimplement the various functions or services to be performed byapplication 204. Application 204 may include, for example, a program forpreventing transition shocks during transitions between realities (e.g.,transitioning between virtual reality and augmented realityenvironments) as discussed further below in connection with FIGS. 3-5.

Referring again to FIG. 2, read-only memory (“ROM”) 205 is coupled tosystem bus 202 and includes a basic input/output system (“BIOS”) thatcontrols certain basic functions of system controller 104. Random accessmemory (“RAM”) 206 and disk adapter 207 are also coupled to system bus202. It should be noted that software components including operatingsystem 203 and application 204 may be loaded into RAM 206, which may besystem controller's 104 main memory for execution. Disk adapter 207 maybe an integrated drive electronics (“IDE”) adapter that communicateswith a disk unit 208, e.g., disk drive.

System controller 104 may further include a communications adapter 209coupled to bus 202. Communications adapter 209 interconnects bus 202with an outside network (e.g., network 105 of FIG. 1) thereby enablingsystem controller 104 to communicate with other devices.

I/O devices may also be connected to system controller 104 via a userinterface adapter 210 and a display adapter 211. Keyboard 212, mouse 213and speaker 214 may all be interconnected to bus 202 through userinterface adapter 210. A display monitor 215 may be connected to systembus 202 by display adapter 211. In this manner, a user is capable ofinputting to system controller 104 through keyboard 212 or mouse 213 andreceiving output from system controller 104 via display 215 or speaker214. Other input mechanisms may be used to input data to systemcontroller 104 that are not shown in FIG. 2, such as display 215 havingtouch-screen capability and keyboard 212 being a virtual keyboard. It isnoted that the input mechanisms discussed herein in connection with FIG.2 may correspond to user interface device 106 of FIG. 1. Systemcontroller 104 of FIG. 2 is not to be limited in scope to the elementsdepicted in FIG. 2 and may include fewer or additional elements thandepicted in FIG. 2.

The present invention may be a system, a method, and/or a computerprogram product. The computer program product may include a computerreadable storage medium (or media) having computer readable programinstructions thereon for causing a processor to carry out aspects of thepresent invention.

The computer readable storage medium can be a tangible device that canretain and store instructions for use by an instruction executiondevice. The computer readable storage medium may be, for example, but isnot limited to, an electronic storage device, a magnetic storage device,an optical storage device, an electromagnetic storage device, asemiconductor storage device, or any suitable combination of theforegoing. A non-exhaustive list of more specific examples of thecomputer readable storage medium includes the following: a portablecomputer diskette, a hard disk, a random access memory (RAM), aread-only memory (ROM), an erasable programmable read-only memory (EPROMor Flash memory), a static random access memory (SRAM), a portablecompact disc read-only memory (CD-ROM), a digital versatile disk (DVD),a memory stick, a floppy disk, a mechanically encoded device such aspunch-cards or raised structures in a groove having instructionsrecorded thereon, and any suitable combination of the foregoing. Acomputer readable storage medium, as used herein, is not to be construedas being transitory signals per se, such as radio waves or other freelypropagating electromagnetic waves, electromagnetic waves propagatingthrough a waveguide or other transmission media (e.g., light pulsespassing through a fiber-optic cable), or electrical signals transmittedthrough a wire.

Computer readable program instructions described herein can bedownloaded to respective computing/processing devices from a computerreadable storage medium or to an external computer or external storagedevice via a network, for example, the Internet, a local area network, awide area network and/or a wireless network. The network may comprisecopper transmission cables, optical transmission fibers, wirelesstransmission, routers, firewalls, switches, gateway computers and/oredge servers. A network adapter card or network interface in eachcomputing/processing device receives computer readable programinstructions from the network and forwards the computer readable programinstructions for storage in a computer readable storage medium withinthe respective computing/processing device.

Computer readable program instructions for carrying out operations ofthe present invention may be assembler instructions,instruction-set-architecture (ISA) instructions, machine instructions,machine dependent instructions, microcode, firmware instructions,state-setting data, or either source code or object code written in anycombination of one or more programming languages, including an objectoriented programming language such as Smalltalk, C++ or the like, andconventional procedural programming languages, such as the “C”programming language or similar programming languages. The computerreadable program instructions may execute entirely on the user'scomputer, partly on the user's computer, as a stand-alone softwarepackage, partly on the user's computer and partly on a remote computeror entirely on the remote computer or server. In the latter scenario,the remote computer may be connected to the user's computer through anytype of network, including a local area network (LAN) or a wide areanetwork (WAN), or the connection may be made to an external computer(for example, through the Internet using an Internet Service Provider).In some embodiments, electronic circuitry including, for example,programmable logic circuitry, field-programmable gate arrays (FPGA), orprogrammable logic arrays (PLA) may execute the computer readableprogram instructions by utilizing state information of the computerreadable program instructions to personalize the electronic circuitry,in order to perform aspects of the present invention.

Aspects of the present invention are described herein with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems), and computer program products according to embodiments of theinvention. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer readable program instructions.

These computer readable program instructions may be provided to aprocessor of a general purpose computer, special purpose computer, orother programmable data processing apparatus to produce a machine, suchthat the instructions, which execute via the processor of the computeror other programmable data processing apparatus, create means forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks. These computer readable program instructionsmay also be stored in a computer readable storage medium that can directa computer, a programmable data processing apparatus, and/or otherdevices to function in a particular manner, such that the computerreadable storage medium having instructions stored therein comprises anarticle of manufacture including instructions which implement aspects ofthe function/act specified in the flowchart and/or block diagram blockor blocks.

The computer readable program instructions may also be loaded onto acomputer, other programmable data processing apparatus, or other deviceto cause a series of operational steps to be performed on the computer,other programmable apparatus or other device to produce a computerimplemented process, such that the instructions which execute on thecomputer, other programmable apparatus, or other device implement thefunctions/acts specified in the flowchart and/or block diagram block orblocks.

The flowchart and block diagrams in the Figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods, and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof instructions, which comprises one or more executable instructions forimplementing the specified logical function(s). In some alternativeimplementations, the functions noted in the block may occur out of theorder noted in the figures. For example, two blocks shown in successionmay, in fact, be executed substantially concurrently, or the blocks maysometimes be executed in the reverse order, depending upon thefunctionality involved. It will also be noted that each block of theblock diagrams and/or flowchart illustration, and combinations of blocksin the block diagrams and/or flowchart illustration, can be implementedby special purpose hardware-based systems that perform the specifiedfunctions or acts or carry out combinations of special purpose hardwareand computer instructions.

As stated in the Background section, currently, when a user transitionsbetween different realities, such as between the virtual reality and theaugmented reality environments or between the virtual reality and thereal reality environments, the user may experience disorientation,motion sickness and/or nauseousness thereby degrading the user'sexperience.

The principles of the present invention provide a means for preventingtransition shocks (e.g., disorientation, nauseousness, motion sickness)during transitions between realities (e.g., transitioning betweenvirtual reality and augmented reality environments) as discussed belowin connection with FIGS. 3-5. FIG. 3 is flowchart of a method forstoring information pertaining to user actions and experiences intransitioning between realities which will be used to determine thelikelihood of a transition shock event occurring. FIG. 4 is a flowchartof a method for preventing transition shocks during transitions betweenrealities. FIG. 5 illustrates a record detailing the modification of thedistance of the object(s) from the reference point in the first realityenvironment and/or the second reality environment so that the user willbe less likely to experience a transition shock.

As discussed above, FIG. 3 is a flowchart of a method 300 for storinginformation pertaining to user actions and experiences in transitioningbetween realities in accordance with an embodiment of the presentinvention.

Referring to FIG. 3, in conjunction with FIGS. 1-2, in step 301, objecttracking system 108 and smart wearable 107 monitor user actions andexperiences in transitioning between realities (e.g., transitioningbetween virtual reality and augmented reality environments,transitioning between virtual reality and real reality environments).Actions, as used herein, involve the movement of user 101, including themovement of hands, arms, legs, feet, head and torso. Such actions mayinvolve physical reactions experienced during the transitions betweenrealities as well as contextual reactions experienced during thetransitions between realities. Experiences, as used herein, involve theuser's response (e.g., symptoms) to transitioning between realities,which may be deduced by monitoring the user's heart rate, the user'sbody temperature, levels of a user's sweat, etc. via smart wearable 107.In one embodiment, such information acquired by object tracking system108, smart wearable 107 is provided to system controller 104.

In step 302, object tracking system 108 and system controller 104determine the changes in the distances between the objects (includingthe distance among the objects as well as the distance between theobjects and a reference object) as well as the speed of transitionsduring the transitions between realities. In one embodiment, thedistance between the objects (e.g., avatar) displayed in the environment(e.g., virtual reality/augmented reality/real reality environment) aswell as between the objects and a reference object (also referred toherein as the “reference point”), such as camera system 109, aredetermined from camera system 109 in conjunction with an objectrecognition service, such as OpenCV. For example, in one embodiment,images of objects are captured by camera system 109. The object in theimage is recognized using the object recognition service. In oneembodiment, the coordinate of the recognized object is determined byreconstructing the depth of the real reality environment (including whenthe real reality environment is being used to generate the virtualreality or augmented reality environment) from the point of view ofcamera system 109. Various “shape from X” techniques (e.g., shape fromstereo, motion, shading or texture) may be used to generate the depthimage from the point of view of camera system 109. A depth map stores,for each pixel in the real-world image, the distance from the XY (image)plane of camera system 109 (reference object) to the real-world surfaceprojected into that pixel. In this manner, the location (coordinate) ofthe objects as well as the distance between the objects (including thedistance between the objects and a reference object, such as camerasystem 109) can be determined. Other embodiments for determining thedistance between the objects include using triangle similarity. Changesin such locations and distances when user 101 transitions betweenrealities may then be determined by determining the differences in thelocation (coordinate) of the objects and the differences in thedistances between the objects before and after user 101 transitionsbetween realties.

In one embodiment, the speed of transitions during the transitionsbetween realities is determined by system controller 104 (e.g.,time-tracking application, such as application 204) tracking thedifference in time between the time user 101 initiates the transitionand the time the transition is completed. In one embodiment, theinitiation of a transition may be user-specified and the completion ofthe transition may be indicated by headset 102. In one embodiment, thecompletion of the transition may be indicated by system controller 104identifying an exit or end activity that indicates the completion of thetransition. In one embodiment, such exit or end activity may be taggedindicating the completion of the transition.

In step 303, system controller 104 maps the monitored user experience intransitioning between realities with the determined changes in thedistances between the objects (including the distance among the objectsas well as the distance between the objects and a reference object) aswell as the speed of transitions during the transitions betweenrealities. For example, system controller 104 maps the user experience(e.g., increased heart rate) in transitioning between the augmentedreality and virtual reality environments with the change in thecoordinate of the avatar object from 332, 455, 233 to 334, 456, 234 andthe speed of the transition (e.g., fifteen seconds). Such informationmay be used by system controller 104 to determine the likelihood of atransition shock event occurring when the user transitions from a firstreality (e.g., augmented reality environment) to a second reality (e.g.,virtual reality environment). For example, if it can be determined thatuser 101 likely experienced a transition shock event (e.g., nausea dueto an excessively high heart rate) when transitioning from the augmentedreality environment to the virtual reality environment when thetransition speed was fifteen seconds along with the avatar objectchanging coordinates from 332, 455, 233 to 334, 456, 234, then it may beinferred that user 101 would experience similar adverse symptoms undersimilar conditions. As discussed further below, if system controller 104determines that user 101 will make a similar transition under a similarcondition, system controller 104 may modify the distances between theobjects (e.g., modify the distance between the object(s) and a referenceobject) when transitioning from the first reality to the second realityin an attempt to prevent user 101 from experiencing a transition shockevent. Such mapped information may be stored in a data storage device(e.g., memory 205, disk unit 208) or in database 110.

In connection with the mapping discussed above, in one embodiment,system controller 104 may also generate models (statistical models) ofprevious transition events between realities and the speed of thetransition events in transitioning between the realities in relation tomotion in a physical space. “Motion,” as used herein, involves themotion or movement of objects during the transition between realitieswhich is tracked by object tracking system 108. The relationship betweenthe motion of objects during the transition between realities and thespeed of the transition event may be statistically modeled. Such modelsmay be stored in a data storage device (e.g., memory 205, disk unit 208)or in database 110.

In step 304, system controller 104 stores the monitored user actionsduring transitions between realities, such as in a data storage device(e.g., memory 205, disk unit 208) or in database 110.

Information pertaining to user actions and experiences in transitioningbetween realities will be used to determine the likelihood of atransition shock event occurring as discussed below in connection withFIG. 4.

FIG. 4 is a flowchart of a method 400 for preventing transition shocksduring transitions between realities in accordance with an embodiment ofthe present invention.

Referring to FIG. 4, in conjunction with FIGS. 1-3, in step 401, objecttracking system 108 monitors user actions. As discussed above, actions,as used herein, involve the movement of user 101, including the movementof hands, arms, legs, feet, head and torso.

In step 402, a determination is made by system controller 104 as towhether user 101 is likely to transition between realities (between thefirst and second realities). For example, system controller 104determines whether the likelihood of user 101 transitioning betweenrealities exceeds a threshold value. For instance, system controller 104may develop a statistical model based on the stored user actions intransitioning between realities to assess a likelihood of whether user101 is likely to transition between realities based on the monitoreduser actions of step 401. For example, the statistical model may includeprobabilities for transitioning between realities when user 101 preformsvarious actions. Such actions may be compared with the current monitoreduser actions (step 401) to determine a likelihood of transitioningbetween realities. Such a likelihood corresponds to a probability valuewhich is compared to a threshold value.

In another embodiment, system controller 104 determines whether user 101is likely to transition between realities based on the user's calendar.For example, system controller 104 may have access to the user'scalendar application (e.g., application 204) which indicates when user101 will be participating in the VR or AR environment. In such anexample when the user is scheduled to participate in the VR or ARenvironment, system controller 104 would assess a probability value ofuser 101 transitioning between realities that exceeds the thresholdvalue.

In another embodiment, system controller 104 determines when user 101 islikely to transition between realities based on feedback provided to anartificial intelligence application (e.g., application 204) running onsystem controller 104. User 101 may continually interact with theartificial intelligence application and provide feedback, such thatbased on certain commands, the artificial intelligence application candetermine that user 101 is likely transitioning to another reality. Insuch an example, system controller 104 would assess a probability valueof user 101 transitioning between realities to exceed the thresholdvalue.

If system controller 104 does not determine that user 101 is likely tobe transitioning between realities, then object tracking system 108continues to monitor user actions in step 401.

If, however, system controller 104 determines that user 101 is likely tobe transitioning between realities (between the first and secondrealities), then, in step 403, system controller 104 determines alocation of objects in the first and second realities. In oneembodiment, the location of objects in the first reality is determinedusing object tracking system 108 as discussed above in step 302. In oneembodiment, the location of the same objects in the second reality isdetermined using statistical analysis. Such statistical analysisinvolves analyzing the stored data (e.g., stored in database 110)pertaining to object locations in the second reality. For example, ifthe avatar object named Mercury is positioned at coordinates 334, 422,233; 334, 412, 212; and 334, 423, 321 during the times user 101 hastransitioned to the augmented reality environment from the virtualreality environment, then it may be determined statistically (e.g.,using directional statistics) that Mercury will likely be positioned at334, 422, 222 when user 101 transitions to the augmented realityenvironment.

In step 404, system controller 104 determines a distance of the objectsfrom a reference point in the first and second realities. That is,system controller 104 determines the distance of the object's locationin the first reality environment from a reference point (e.g., camerasystem 109) as well as determines the distance of the object's locationin the second reality environment from the reference point. The object'slocation (e.g., coordinate) in the first and second reality environmentsare determined in step 403. In one embodiment, the distance of theobject from the reference point is determined based on the expected timeof interaction (i.e., the time of interaction with the environment, suchas the virtual reality environment). In one embodiment, the distance ofthe object from the reference point is determined based on the movementof objects in relation to each other. In one embodiment, the distance ofthe object from the reference point is determined based on the taggedobject distances. For example, objects may be tagged with a distancebetween another object which may be used to determine the distance ofthe object from the reference point.

In step 405, system controller 104 determines the likelihood of atransition shock event occurring as the user transitions between thefirst reality environment to the second reality environment. A“transition shock” or a “transition shock event,” as used herein, refersto the occurrence of adverse symptoms, such as disorientation,nauseousness or motion sickness, experienced by user 101 when user 101transitions between different realties. In one embodiment, thelikelihood of the transition shock event occurring is ascertained basedon one or more of the following: historical occurrences of symptomsexperienced during transitions between realities (discussed inconnection with FIG. 3), historical occurrences of physical reactionsexperienced during transitions between realities (discussed inconnection with FIG. 3), modeling of previous transition events betweenrealities (discussed in connection with FIG. 3) and speed of transitionevents in transitioning between realities in relation to motion in aphysical space (discussed in connection with FIG. 3). In one embodiment,such information is stored in database 110. In one embodiment, thelikelihood of the transition shock event occurring is ascertained basedon the location (coordinate) and/or distance of the object from thereference point in the first and second reality environments.

For example, suppose that user 101 experienced a high heart rateindicating nausea (historical occurrence of symptom) along with adownward movement of the head indicating nausea (historical occurrenceof physical reaction) when user 101 transitions from the virtual realityenvironment to the augmented reality environment when the transitionspeed was fifteen seconds along with the avatar object changingcoordinates from 332, 455, 233 to 334, 456, 234. A probability (e.g.,value ranging from 0 to 1, with 1 indicating the event will definitelyoccur and 0 indicating the event will definitely not occur) that atransition shock event will occur is assessed by system controller 104using the determined location of the objects in the first and secondrealities (determined in step 403) and the determined distance of theobjects from a reference point in the first and second realities(determined in step 404). For example, if user 101 is likely totransition between the virtual reality environment to the augmentedreality environment under similar conditions as in the example discussedabove (e.g., similar transition speed with similar coordinates of theavatar object), then it may be inferred that user 101 would likelyexperience similar adverse symptoms (i.e., system controller 104 wouldassess a high probability that a transition shock event is likely tooccur).

In step 406, a determination is made as to whether the likelihood of atransition shock occurring exceeds a threshold value, which may beuser-selected. In one embodiment, the probability value ascertained instep 405 is compared with the threshold value. If the probability valuedoes not exceed the threshold value, then, in step 407, the distances ofthe objects from the reference point in the first and/or second realityenvironment are not modified by system controller 104.

If, however, the probability value exceeds the threshold value, then, instep 408, camera system 109 and system controller 104 capture the imagesof the objects in the first and second realities, where the images ofthe objects in the first reality environment are captured by camerasystem 109 (and sent to system controller 104) and the images of thesame objects to be presented in the second reality environment aregenerated by system controller 104 based on the determined location ofthe same objects in the second reality environment as determined in step403.

In step 409, system controller 104 sends the captured images to anobject recognition service, such as OpenCV. The object in the image isrecognized using the object recognition service.

In step 410, system controller 104 modifies the distance of theobject(s) from the reference point in the first reality and/or thesecond reality so that the user will less likely experience a transitionshock. For example, the distance of the object(s) from the referencepoint in the first reality and/or the second reality is modified basedon the position of future scenes and depths and specific user cohorts(e.g., a user of the video game of Minecraft® may have a differentsusceptibility to transition shock than an architect, where suchsusceptibility is determined based on historical experiences oftransition shock during transitions between different realities).

In one embodiment, the distance of the object(s) from the referencepoint in the first reality and/or the second reality is modified basedon previous locations of the objects and/or distances of the objectsfrom the reference point where no transition shock was experienced byuser 101. In one embodiment, based on previous transitions betweenrealities, a model is generated by system controller 104 detailinglocations of the objects and/or distances of the objects from thereference point where transition shocks occur and do not occur. Such amodel may be used by system controller 104 to modify the distance of theobject(s) from the reference point in the first reality and/or thesecond reality so that the user will less likely experience a transitionshock. To be clear, modifying the distance of the object(s) from thereference point involves changing the location (coordinate) of theobjects.

A record detailing such a modification is shown in FIG. 5. FIG. 5illustrates a record detailing the modification of the distance of theobject(s) from the reference point in the first reality environmentand/or the second reality environment so that the user will be lesslikely to experience a transition shock in accordance with an embodimentof the present invention.

Referring to FIG. 5, record 500 includes the context 501 (e.g., eventmanagement), the classification 502 (e.g., information technology),likely transition 503 (e.g., VR to AR), transition time 504 (e.g., 239seconds), focus object 505 (e.g., laptop1), relative distance (current)506 (e.g., 6 meters), relative distance (future reality) 507 (e.g., 2meters), recommendation 508 (e.g., transition object), transition phase509 (e.g., before (1 meter) AND after (3 meters), sensitivity 510 (e.g.,245) and userID 511 (e.g., user59). Focus object 505 may refer to thereference point and the current relative distance 506 refers to thecurrent distance of the object from the reference point and the futurerelative distance 507 refers to the distance of the object from thereference point in the future reality environment. Recommendation 508refers to whether or not it is recommended to transition the object.Transition phase 509 refers to the changes in the distance of the objectfrom the reference point that need to occur in order for the location ofthe object to have the desired distance from the reference point.Sensitivity 510 refers to a number that is assigned by system controller104 detailing the level of sensitivity or likelihood of exhibiting atransition shock. In one embodiment, the higher the number, the moresensitive or more likely user 101 will experience a transition shock. Inone embodiment, the level of sensitivity is determined by systemcontroller 104 based on how close are the locations of the objects andthe distances of the objects from the reference point to those locationsand distances that caused user 101 to experience a transition shock whentransitioning between realities. UserID 511 refers to an identificationassigned to user 101.

In this manner, user 101 may be prevented from experiencing transitionshocks (e.g., disorientation, nauseousness, motion sickness) whentransitioning between realities (e.g., transitioning between the virtualreality and augmented reality environments). As a result, users will beable to participate in several different reality systems (e.g., virtualreality, augmented reality, real reality), one after the other, withoutexperiencing transition shocks. That is, the technical problem of havingdifferent reality systems being able to be utilized by the user, oneafter the other, is addressed by the present invention. As discussedabove, the present invention provides a technological solution to thetechnical problem by preventing users from experiencing transitionshocks when transitioning between realities thereby encouraging userinteractions between different reality systems, such as virtual realityand augmented reality systems. That is, integration andinterchangeability between different reality systems is improved.

The descriptions of the various embodiments of the present inventionhave been presented for purposes of illustration, but are not intendedto be exhaustive or limited to the embodiments disclosed. Manymodifications and variations will be apparent to those of ordinary skillin the art without departing from the scope and spirit of the describedembodiments. The terminology used herein was chosen to best explain theprinciples of the embodiments, the practical application or technicalimprovement over technologies found in the marketplace, or to enableothers of ordinary skill in the art to understand the embodimentsdisclosed herein.

The invention claimed is:
 1. A method for transitioning betweendifferent realities, the method comprising: mapping monitored userexperience in transitioning between different realities with determinedchanges between objects as well as speed of transitions duringtransitions between different realities; storing monitored actions of auser during transitions between realities, wherein said monitoredactions comprise movement of hands, arms, legs, feet, head and torso;developing a statistical model based on said stored monitored useractions in transitioning between realities to assess a probability ofsaid user transitioning between realities; determining whether aprobability of said user transitioning between a first realityenvironment and a second reality environment exceeds a first thresholdvalue based on current monitored user actions and said statisticalmodel; determining a location of an object in said first realityenvironment and said second reality environment in response todetermining that said probability of said user transitioning betweensaid first reality environment and said second reality environmentexceeds said first threshold value; determining a distance of saidobject from a reference point in said first reality environment and insaid second reality environment using said determined location of saidobject in said first and second reality environments; ascertaining, by aprocessor, whether a probability of a transition shock event occurringexceeds a second threshold value as said user transitions between saidfirst and second reality environments using said mapped monitored userexperience, wherein said second threshold value is different from saidfirst threshold value, wherein said transition shock event is anoccurrence of adverse symptoms experienced by said user as said usertransitions between said first and second reality environments; andmodifying, by said processor, said distance of said object from saidreference point in one or more of said first reality environment andsaid second reality environment in response to ascertaining that saidprobability of said transition shock event occurring exceeds said secondthreshold value.
 2. The method as recited in claim 1, wherein saidprobability of said transition shock event occurring is ascertainedbased on one or more of the following: historical occurrences ofsymptoms experienced during transitions between realities, historicaloccurrences of physical reactions experienced during transitions betweenrealities, modeling of previous transition events between realities andspeed of transition events in transitioning between realities inrelation to motion in a physical space.
 3. The method as recited inclaim 1, wherein said distance of said object from said reference pointis determined by one of the following: expected time of interaction,movement of objects in relation to each other and tagged objectdistances.
 4. The method as recited in claim 1, wherein said distance ofsaid object from said reference point in one or more of said firstreality environment and said second reality environment is modifiedbased on the following: position of future scenes and depths in otherrealities and specific user cohorts.
 5. The method as recited in claim 1further comprising: capturing an image of said object in said first andsecond reality environments in response to ascertaining that saidprobability of said transition shock event occurring exceeds said secondthreshold value; and sending said captured images to an objectrecognition service.
 6. The method as recited in claim 1, wherein saidprobability of said transition shock event occurring is ascertainedbased on location and said distance of said object from said referencepoint in said first and second reality environments.
 7. The method asrecited in claim 1, wherein said first and second reality environmentscomprise one of the following: an augmented reality environment, avirtual reality environment and a real reality environment, wherein saidfirst and second reality environments are different realityenvironments.
 8. A computer program product for transitioning betweendifferent realities, the computer program product comprising anon-transitory computer readable storage medium having program codeembodied therewith, the program code comprising the programminginstructions for: mapping monitored user experience in transitioningbetween different realities with determined changes between objects aswell as speed of transitions during transitions between differentrealities; storing monitored actions of a user during transitionsbetween realities, wherein said monitored actions comprise movement ofhands, arms, legs, feet, head and torso; developing a statistical modelbased on said stored monitored user actions in transitioning betweenrealities to assess a probability of said user transitioning betweenrealities; determining whether a probability of said user transitioningbetween a first reality environment and a second reality environmentexceeds a first threshold value based on current monitored user actionsand said statistical model; determining a location of an object in saidfirst reality environment and said second reality environment inresponse to determining that said probability of said user transitioningbetween said first reality environment and said second realityenvironment exceeds said first threshold value; determining a distanceof said object from a reference point in said first reality environmentand in said second reality environment using said determined location ofsaid object in said first and second reality environments; ascertainingwhether a probability of a transition shock event occurring exceeds asecond threshold value as said user transitions between said first andsecond reality environments using said mapped monitored user experience,wherein said second threshold value is different from said firstthreshold value, wherein said transition shock event is an occurrence ofadverse symptoms experienced by said user as said user transitionsbetween said first and second reality environments; and modifying saiddistance of said object from said reference point in one or more of saidfirst reality environment and said second reality environment inresponse to ascertaining that said probability of said transition shockevent occurring exceeds said second threshold value.
 9. The computerprogram product as recited in claim 8, wherein said probability of saidtransition shock event occurring is ascertained based on one or more ofthe following: historical occurrences of symptoms experienced duringtransitions between realities, historical occurrences of physicalreactions experienced during transitions between realities, modeling ofprevious transition events between realities and speed of transitionevents in transitioning between realities in relation to motion in aphysical space.
 10. The computer program product as recited in claim 8,wherein said distance of said object from said reference point isdetermined by one of the following: expected time of interaction,movement of objects in relation to each other and tagged objectdistances.
 11. The computer program product as recited in claim 8,wherein said distance of said object from said reference point in one ormore of said first reality environment and said second realityenvironment is modified based on the following: position of futurescenes and depths in other realities and specific user cohorts.
 12. Thecomputer program product as recited in claim 8, wherein the program codefurther comprises the programming instructions for: capturing an imageof said object in said first and second reality environments in responseto ascertaining that said probability of said transition shock eventoccurring exceeds said second threshold value; and sending said capturedimages to an object recognition service.
 13. The computer programproduct as recited in claim 8, wherein said probability of saidtransition shock event occurring is ascertained based on location andsaid distance of said object from said reference point in said first andsecond reality environments.
 14. The computer program product as recitedin claim 8, wherein said first and second reality environments compriseone of the following: an augmented reality environment, a virtualreality environment and a real reality environment, wherein said firstand second reality environments are different reality environments. 15.A system, comprising: a memory unit for storing a computer program fortransitioning between different realities; and a processor coupled tothe memory unit, wherein the processor is configured to execute theprogram instructions of the computer program comprising: mappingmonitored user experience in transitioning between different realitieswith determined changes between objects as well as speed of transitionsduring transitions between different realities; storing monitoredactions of a user during transitions between realities, wherein saidmonitored actions comprise movement of hands, arms, legs, feet, head andtorso; developing a statistical model based on said stored monitoreduser actions in transitioning between realities to assess a probabilityof said user transitioning between realities; determining whether aprobability of said user transitioning between a first realityenvironment and a second reality environment exceeds a first thresholdvalue based on current monitored user actions and said statisticalmodel; determining a location of an object in said first realityenvironment and said second reality environment in response todetermining that said probability of said user transitioning betweensaid first reality environment and said second reality environmentexceeds said first threshold value; determining a distance of saidobject from a reference point in said first reality environment and insaid second reality environment using said determined location of saidobject in said first and second reality environments; ascertainingwhether a probability of a transition shock event occurring exceeds asecond threshold value as said user transitions between said first andsecond reality environments using said mapped monitored user experience,wherein said second threshold value is different from said firstthreshold value, wherein said transition shock event is an occurrence ofadverse symptoms experienced by said user as said user transitionsbetween said first and second reality environments; and modifying saiddistance of said object from said reference point in one or more of saidfirst reality environment and said second reality environment inresponse to ascertaining that said probability of said transition shockevent occurring exceeds said second threshold value.
 16. The system asrecited in claim 15, wherein said probability of said transition shockevent occurring is ascertained based on one or more of the following:historical occurrences of symptoms experienced during transitionsbetween realities, historical occurrences of physical reactionsexperienced during transitions between realities, modeling of previoustransition events between realities and speed of transition events intransitioning between realities in relation to motion in a physicalspace.
 17. The system as recited in claim 15, wherein said distance ofsaid object from said reference point is determined by one of thefollowing: expected time of interaction, movement of objects in relationto each other and tagged object distances.
 18. The system as recited inclaim 15, wherein said distance of said object from said reference pointin one or more of said first reality environment and said second realityenvironment is modified based on the following: position of futurescenes and depths in other realities and specific user cohorts.
 19. Thesystem as recited in claim 15, wherein the program instructions of thecomputer program further comprise: capturing an image of said object insaid first and second reality environments in response to ascertainingthat said probability of said transition shock event occurring exceedssaid second threshold value; and sending said captured images to anobject recognition service.
 20. The system as recited in claim 15,wherein said probability of said transition shock event occurring isascertained based on location and said distance of said object from saidreference point in said first and second reality environments.